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ANNUAL  TECHNICAL  REPORT 


AFOSR  Contract  No.  F49620-82-K-0014 
Submitted  by  New  York  University 


INTRODUCTION 


This  annual  technical  report  describes  the  progress  we  have 
made  in  studying  "The  E-ffects  o-f  Workload  and  Attention  on  the 
Neuromagnetic  Response."  Using  a  system  based  on  the 
Superconducting  Quantum  Inter-ference  Device  (SQUID),  we  are  able 
to  detect  extremely  weak  magnetic  -fields  associated  with  the  -flow 
of  electric  current  in  the  human  brain.  Because  of  unique 
features  of  this  method,  it  is  possible  to  sharply  localize  the 
source  of  the  field.  In  fact,  the  starting  point  of  this  project 
was  our  success  in  locating  the  source  of  the  'P300’  component  of 
the  Event  Related  Potential  (ERP)  in  or  near  the  hippocampal 
formation.  This  led  to  an  attempt  to  identify  cortical  sources 
of  components  of  the  ERP,  and  to  determine  whether  the  variation 
in  activity  of  sources  that  contribute  to  the  ERP  varies 
differentially  with  the  nature  of  the  task  required  of  the 
subject,  as  well  as  his  or  her  state  of  attention.  We  have  made 
progress  on  several  fronts  during  this  project.  This  report  will 
describe  developments  in  several  areas,  and  indicate  how  they 

relate  to  the  overall  goals  of  the  project. 
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During  the  first  year  of  the  project  we  worked  with  a  magnetic 
sensing  system  based  on  the  rfSQUID.  During  that  time  the  S.H.E. 
Corp.  announced  a  commercial  version  of  a  dcSQUID  which  provides 
an  improvement  of  an  order — of-magnitude  in  intrinsic.  SQUID  noise. 
Of  course,  noise  due  to  other  components  of  the  system  also 
contribute,  but  by  incorporating  a  dcSQUID,  as  well  as  other 
changes,  e.g.,  to  the  rf  shield,  we  anticipated  a  gain  in 
signal-to-noise  by  a  factor  4.  These  modifications  were  made,  and 
after  extensive  tests  we  found  that  the  upgraded  system  had  a 
white-noise 'level  of  20  fT  per  root  Hertz,  which  is  a  factor  of  4 
better  than  the  prior  system.  This  made  it  possible  for  us  to 
obtain  more  reliable  measures  of  fields  associated  with  the  ERP. 

It  also  allowed  us  to  make  some  tests  that  affected  the  design  of 
the  multi -sensor  array  to  be  described  below. 

In  one  of  these  tests  we  used  an  rfSQUID  to  sense  the  ambient 
noise,  and  used  this  noise  to  cancel  correlated  1/F  noise  sensed 
by  the  dcSQUID.  This  test  was  necessary  to  determine  if  it  is 
feasible  to  reduce  the  1/F  noise  (which  becomes  appreciable  below 
about  1  Hz  in  the  dcSQUID  system).  Since  many  of  the  ERP 
components  of  interest  have  significant  low-f reguency  content  (and 
are  distfsrted  by  filter  cut-offs  of  0.5  Hz  or  higher),  it  is  of 
some  importance  to  reduce  the  low-frequency  1/F  noise.  This 
cancellation  procedure  was  found  to  be  highly  effective,  and  we 
decided  to  incorporate  it  in  the  multi -sensor  array. 


2.  The  Multi -Sensor  Array 

With  funding  from  the  Office  of  Naval  Research,  we  worked  with 
S.H.E.  Corp.  scientists  in  developing  a  prototype  system  that 
includes  5  channels  based  on  dcSQUIDs,  plus  4  rfSQUID  channels 
for  sensing  the  ambient  field  for  purposes  of  noise  reduction. 
Three  of  the  rfSQUIOs  are  configured  as  magnetometers  and  sense 
field  in  the  X,  Y  and  Z  directions.  The  fourth  rfSQUID  is  part  of 
a  first-order  gradiometer  system  to  sense  the  field  gradient  in 
the  Z-axis.  These  four  outputs,  together  with  their 
time— derivatives,  will  be  given  optimum  weightings  before  being 
used  to  cancel  noise  in  the  active  ch'annels  (second  order 
gradiometers  coupled  to  dtSQUIDs) .  The  system  has  undergone 
preliminary  tests  at  S.H.E.  Carp,  and  meets  all  specifications. 

We  expect  to  take  delivery  within  the  next  7  days. 

While  ONR  covered  the  cost  of  this  system  and  associated 
electronics,  we  expended  considerable  effort  on  the  present 
program  in  conducting  preliminary  tests,  designing  and 
constructing  a  flexible  holder  for  positioning  the  large  dewar 
and  for  scanning  the  head,  and  in  preparing  software  for  the  use 
of  the  system  in  this  workload  study.  The  scanner  is  now 
cofMsIeted  and  is  in  the  laboratory  awaiting  the  multi— sensor 
system.  It  is  being  oufitted  with  stepping  motors  so  that 
ultimately  it  can  be  placed  under  computer-control  for  scanning 
the  head.  One  of  the  main  justifications  for  this  extensive 
effort  is  that  repeated  measurements  from  about  80  positions 
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•bout  the  head  were  required  to  map  the  field  pattern  associated 
with  the  P300  component.  This  was  an  extremely .time-consuming 
task.  Moreover,  even  in  a  simple  odd— ball  experiment  it  is 
difficult  to  assure  that  the  subject's  response  would  remain 
stable  over  many  hundreds  of  repeated  trials.  Since  there  will 
be  five  active  channels  in  the  new  system,  it  should  be  possible 
to  map  the  field  much  more  quickly,  and  require  far  fewcn^  trials. 
Also  the  enhanced  si gnal -to— noi se  ratio  will  allow  us  to  use 
fewer  replications.  In  effect,  there  is  a  built  in  replication 
because  of  the  configuration  of  pick-up  coils,  which  will  allow 
us  to  keep  a  coil  over  one  spot  while  the  other  four  coils  are 
moved  to  new  spots. 

The  five  channels  for  measuring  fields  from  the  brain,  as 
well  as  8  channels  of  ambient  field  data  for  cancellation  of 
noise  by  software,  required  us  to  develop  an. extensive  package  of 
ccxnputer  programs  over  the  past  few  months.  In  addition,  the 
amount  of  data  we  will  have  to  analyze  will  be  truly  enormous,  so 
arrangements  were  made  to  allow  our  PDPil/34  computer  to 
communicate  with  the  mainframe  CYBER  computer  or  with  one  of  the 
VAX  computers  at  the  Courant  Institute.  This  computer  time  will 
be  provided  at  no-cost  to  the  project,  and  it  will  facilitate  the 
extensive  "number  crunching"  that  will  be  associated  with  every 
experiment.  With  all  of  this  preparatory  work  completed  or 
nearing  completion,  it  should  require  a  minimum  amount  of  time  to 
get  the  new  system  up  and  running. 


II.  EXPERIMENTAL  RESULTS 

1.  Modulation  Studies 

In  1970  Kaufman  and  Lfscker  reported  that  EEG  activity  is 
modulated  in— step  with  event  related  potentials.  Also, 

Spekreisje  and  Reits  (1980)  demonstrated  the  presence  of 
sidebands  in  the^EEG  spectrum  during  sensory  stimulation.  TYiese 
sidebands  must  result  in  modulation  of  the  EES. 

The  aiodulation  of  the  EEG  by  event  related  potentials  is 
bound  to  occur  if  spontaneous  EEG  activity  and  evoked  activity 
cx:cur  together  in  the  same  (nonlinear)  neural  structures.  Since 
evoked  cortical  activity  is  often  at  twice  the  frequency  of  any 
simple  harmonic  input,  cortical  neural  networks  must  often 
exhibit  essential  nonlinearities. 

It  is  important  to  recognize  that  the  averaging  computer  is  a 
comb  filter,  in  the  sense  that  it  enhances  harmonics  of  the 
fundamental  stimulus  frequency  and  attenuates  the  background 
"noise".  Thus,  all  commponents  of  the  EEG  that  are  not 
time-locked  to  a  stimulus  event  are  attenuated  as  noise.  This 
would  include  all  of  the  sidebands  resulting  from  any  interaction 
of  the  evoked  activity  and  the  ongoing  EEG.  Hence,  valuable 
information  may  be  discarded  along  with  noise  during  the  signal 
averaging  process.  This  information  may  well  complement  the 
NlOO,  P300  amd  N400,  as  well  as  other  components  of  the  ERP  that 
reflect  mental  activity  of  different  kinds. 


The  fact  that  alpha  blockage  occurs  during  visual  attention 
is  a  strong  indication  that  at  least  some  type  of  change  occurs 
in  the  spontaneous  EEG.  Existing  theories  of  this  blockage  are 
based  on  the  notion  of  "desynchronization",  which  suggests  that 
normal  alpha  is  attributable  to  a  group  of  loosely  coupled 
free— running  oscillators,  and  these  become  uncoupled  during 
attention,  so  that  activity  at  alpha  frequencies  is  effectively 
cancelled  because  the  oscillations  are  out— of— phase  with  each 
other.  An  alternative  view  is  that  alpha  activity  occurring  in 
cortical  neurons  is  modulated  because  of  non-linear  interactions 
with  other  attent ion— rel ated  cortical  events.  While  Kaufman  and 
Locker  demonstrated  modulation  of  brain  activity  by  steady  state 
conditions, -we  focussed  on  its  modulation  by  transient  events  — 
events  that  are  capable  of  evoking  P300-like  phenomena. 

Before  preceding  with  a  sumnmary  of  our  results  to  date,  it  is 
important  to  keep  in  mind  the  distinction  between  cortical  EEG  and 
components  of  the  ERP,  which  may  or  may  not  have  cortical 
sources.  P300  appears  to  have  a  subcortical  source,  namely  in 
the  hippocampal  formation.  NlOO,  which  is  also  affected  by 
cognitive  factors,  seems  to  have  a  cortical  source,  as  does  N400 
(see  below).  Alpha  blockage  (which  we  do  not  consider  to  be  a 
form  of  "desynchronization")  is  most  likely  to  be  related  to 
modulation  of  activity  of  neurons  in  the  visual  cortex,  and  is 
not  necessarily  a  global  effect  of  some  sub-cortical  pacemaker. 

In  fact,  if  it  were  a  more  global  effect,  it  should  appear  during 
attention  to  auditory  stimuli  as  well  as  visual,  an  effect  which 


Me  studied  in  this  report  period. 

One  of  the  most  fascinating  aspects  of  this  project  is  that  it 
provides  us  with  a  method  to  complement  the  usual  method  of 
recording  electric  potentials  at  the  scalp.  These  potentials  are 
widely  distributed  and  problems  exist  in  separating  sources  from 
each  other.  While  similar  problems  still  beset  the  intepretation 
of  neuromagnetic  field  data,  there  is  ample  reason  to  believe 
that  such  data  are  simpler  to  intepret.  In  fact,  one  of  the 
accomplishments  of  our  program  to  date  is  the  finding  that  even 
if  a  source  is  a  sheet  of  current  dipoles  more  than  1  cm  on  a 
side,  the  d-ifference  in  the  computed  location  of  the  "center  of 
gravity”  of  the  sheet  differs  by  no  more  than  1  or  2%  from  the 
computed  location  of  the  source  when  one  assumes  that  it  is  a 
point  current  dipole  at  any  reasonable  depth  beneath  the  scalp. 
Thus,  the  error  attributable  to  the  assumption  of  an  equivalent 
current  dipole  as  the  source  of  the  observed  field  is  smaller 
than  the  experimental  error  associated  with  taking  such 
measuremments.  Therefore,  it  is  quite  reasonable  to  assume  that 
fields  having  different  sources  in  widely  separated  positions  in 
the  brain  can  be  resolved  by  magnetic  measures.  So  one  of  the 
problems  we  are  addressing  is  the  differences  in  locations  of 
sources  of  phenomena  that  may  be  detectable  by  means  of  electrodes 
attached  to  the  scalp.  These  phenomena  include  the  classic 
components  of  the  ERP  as  well  as  the  hypothesized  modulation  of 
the  background  EEG. 

To  test  for  the  presence  of  this  modulation  we  employed  an 


odd  ball  paradigm  in  which  a  frequent  visual  stimulus  was 
occassional  ly  interrupted  by  an  infrequent  (2071)  auditory 
stimulus  and,  in  other  blocks  of  trials,  the  reverse  procedure  was 
used.  Me  measured  the  ERP  betwen  an  electrode  at  the  vertex  or 
at  Pz  (this  varied  across  subjects),  and  one  at  one  mastoid,  with 
the  forehead  as  ground.  Also,  we  measured  the  magnetic  field 
outside  the  head  in  the  vicinity  of  the  temporal  cortex  using  our 
single  dcSQUID  sensor.  We  deliberately  put  off  mapping  studies, 
pending  the  arrival  of  our  multi— sensor  array,  because- it  would 
have  been  uneconomical  to  do  otherwise.  However,  it  is  worth 
noting  that  even  with  the  pick  up  coil  positioned  at  a 
non-optimum  place  (not  at  an  extremum*  of  the  field)  it  was 
possible  to  retrieve  responses  to  odd  balls  with  only  15-20 
trials. 

Our  data  indicate  that  odd  ball  stimuli  (visual  or  auditory) 
usually  (though  not  always)  show  an  elevated  P30p.  One  subject 
in  particular  did  not  show  an  enhancement  of  P300  for  odd  ball 
stimuli.  However,  for  visual  stimuli  all  subjects  showed  a 
depression  in  spontaneous  activity  in  the  alpha  band  immediately 
following  NlOO,  and  this  depression  persisted  for  about  500  msec 
when  the  visual  stimulus  was  an  odd  ball.  If  the  visual  stimulus 
was  frequent,  the  depression  of  alpha  band  activity  was  present 
but  was  quickly  restored  to  baseline  after  about  iOO  msec.  This 
was  an  invariant  effect  across  subjects,  regardless  of  whether  or 
not  P300  showed  the  usual  and  expected  change,  depending  upon 
whether  the  stimulus  was  an  odd  ball  or  not. 
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Brie-fly,  the  procedure  used  to  recover  this  modulation  effect 
Mas  to  store  all  of  the  raw  EEG  data  on  magnetic  tape  and  then 
process  it,  trial  by  trial,  by  first  reducing  all  trials  to  the 
same  length  (they  were  typically  1500  msec  between  stimuli,  but 
randomized  in  duration  by  as  much  as  500  msec),  and  then 
stringing  them  together  so  that  there  was  a  fundamental  frequency 
with  a  period  equal  to  the  duration  of  the  truncated  ISI.  Then 
the  fundamental  frequency  and  all  of  its  harmonics  were  removed 
from  the  trials,  leaving  only  the  "noise"  that  was  not 
harmonically  related  to  the  stimulus.  This  noise  was  then 
rectified,  and  the  rectified  noise  was  averaged.  This  average 
revealed  an  invariant  waveform  for  odd  ball  trials  and  for 
frequent  trials,  but  they  differed  in  shape  and  duration.  When 
the  “noise"  alone  was  averaged  without  rectification,  there  was 
no  response  at  all,  thereby  proving  that  the  background  activity 
was  indeed  modulated  by  the  visual  event,  for  otherwise  any 
residual  stimulus-coherent  activity  would  have  yielded  a 
response. 

It  is  of  some  interest  to  note  that  only  the  visual  stimulus 
was  capable  of  eliciting  this  modulation  wave  (M-wave) .  Neither 
the  odd-ball  nor  the  frequent  auditory  stimulus  produced  such  an 
effect.  However,  there  was  a  difference  in  NlOO  between  auditory 
and  odd  ball  stimuli,  but  this  is  still  being  investigated.  Also, 
other  electrode  placements  are  being  tried,  since  a  comparable 
effect  In  the  auditory  system  may  have  a  source  that  differs  in 
orientation  and/or  location  from  the  location  of  the  set  of 


neurons  that  respond  to  the  auditory  stimulus  per  se.  In  any 
event,  when  the  ''active”  electrode  was  placed  near  Oz,  the  H— wave 
became  much  stronger  (by  a  -factor  of  3  or  4)  than  it  was  with  the 
active  electrode  at  Pz.  This  suggests  a  source  in  the  visual 
cortex  for  an  attention— related  effect  on  the  EEG. 

It  is  quite  interesting  that  the  M— wave  occurs  coincidentally 
with  NlOO  and  is  not  further  modulated  by  P300  or,  when 
encountered,  by  N400.  Thus,  the  effect  seems  to  occur 
independently  of  P300,  and,  unlike  the  latter,  it  does  seem  to 
have  a  cortical  source. 

Having  verified  the  presence  of  an  effect  on  brain  events 
that  are  not  visible  in  the  average  ERP  but  are  nonetheless 
dependent  upon  attention  to  these  events,  we  set  out  to  determine 
if  their  sources  were  different  from  the  sourceiS  of  the  usual 
ERP.  This  was  done  by  means  of  magnetic  measures.  As  indicated, 
these  measures  were  made  in  the  temporal  region,  and  systematic 
mapping  was  not  yet  undertaken.  In  any  event,  for  both  visual 
and  auditory  stimuli  we  detected  P300-like  phenomena  in  the 
temporal  region  and  encountered  no  reversals  in  phase  in  the 
temporal  area.  NlOO  was  detected  when  auditory  stimuli  were 
used,  but  not  when  visual  stimuli  were  used.  Undoubtedly  a 
visual  NlOO  would  have  been  encountered  if  we  had  mapped  the  field 
in  the  occipital  region.  However,  the  main  point  is  that  none  of 
the  measures  made  to  date  in  the  temporal  region  revealed  the 
presence  of  a  magnetic  N-wave.  We  must  yet  place  the  pick  up 
coil  over  the  occipital  region  where  we  will  undoubtedly  sense 


such  an  even  and  locate  its  source.  Tentatively,  the  11-wave  is  a 
modulation  of  activity  of  one  of  the  visual  areas,  and  this 
modulation  rewpresents  an  inhibition  of  spontaneous  activity  when 
visual  stimuli  are  being  processed  by  the  brain.  If  the  incoming 
signal  is  not  "interesting"  (as  in  the  case  of  frequent  stimuli), 
the  inhibition  is  of  limited  duration.  However,  if  the  incoming 
visual  signal  is  an  odd  ball  ("interesting"),  then  the  visual 
cortex  may  well  be  conserving  its  capacity  to  respond  by  not 
returning  to  its  normally  active  state. 

2.  Magnetic  Counteparts  to  NlOO 

As  evidenced  by  the  work  of  Hillyard  and  his  colleagues,  NlOO 
also  displays  variation  in  amplitude  when  cognitive  variables  are 
introduced.  The  magnetic  counterpart  to  NlOO  was  measured  when  an 
acoustic  stimulus  was  applied  to  a  subject.  Th^  source  of  this 
component  was  located  about  1  cm  anterior  to  the  tonotopical ly 
organized  portion  of  the  auditory  cortex  in  2  subjects. 

Moreover,  the  location  of  this  source  did  not  depend  upon  the 
frequency  content  of  the  tone  pip  used  to  evoke  the  response. 
Preliminary  evidence  suggests  that  this  component  is  contingent 
upon  the  presence  of  virtually  any  acoustic  stimulus  (tone  pip, 
click  or  square-wave  change  in  sound  pressure  level),  but  its 
amplitude  depends  instead  upon  the  attention  given  to  the  event. 
Work  is  continuing  on  this  observation.  In  any  event,  one 
relatively  firm  conclusion  is  that  sensory  stimulation  is 


necessary  to  the  occurrence  of  NlOO  (the  missing-stiinulus 
paradigm  doesn't  work)  but  is  not  sufficient  to  account  for  its 
magnitude.  Cognitive  factors  are  the  major  contributors  to  the 
variance  in  the  magnitude  of  NlOO.  This  also  seems  to  be  true  of 
the  visual  NlOO,  but  we  lack  sufficient  data  to  draw  firm 
conclusions. 

3.  Nagnetic  Counterparts  to  N400 


In  some  of  our  earlier  attempts  to  produce  the  P300 
phenomenon  by  means  of  electrical  recording  (in  the  usual 
fashion),  we  ran  into  two  problems.  The  first  was  an  unwanted 
effect  of  eye  movements,  which  we  had  to  eliminate  by  monitoring 
shifts  in  eye  postion  using  electrodes  attached  to  the  face  near 
the  eyes.  Once  this  was  accomplished  we  discovered  that  another 
component  of  the  ERP  often  masked  P300.  This  was  a  component  at 
N400,  which  was  of  a  very  large  amplitude,  and  P300  could  only  be 
observed  as  a  change  in  the  level  of  negativity  of  N40O.  One  of 
our  goals  was  to  find  a  clean  electrical  response,  similar  to 
those  studied  by  other  groups,  so  that  we  could  identify  the 
sources  of  its  components  and  observe  their  variability, 
depending  upon  stimulus  conditions  and  the  state  of  the  subject. 
We  had  knowledge  of  the  work  of  Kutas  and  of  Hillyard  on  N400. 
Kutas  used  either  semantic  or  syntactic  anomalies  in  the  context 


of  a  dichotic  listening  task  to  produce  this  effect.  We  did  not 
expect  to  encounter  it  in  an  odd  ball  task.  Nonetheless,  we  did. 
Since  it  was  present,  we  decided  to  examine  the  magnetic  response 
in  the  temporal  area  to  see  if  there  was  reversal  of  polarity  of 
the  response.  For  if  such  a  reversal  occurred  over  a  short 
distance  across  the  scalp,  it  would  prove  that  N400  (unlike  P300) 
had  a  cortical  source.  In  fact,  we  did  find  such  a  reversal  of 
polarity.  While, the  data  are  still  being  evaluated,  it  is 
possible  to  say  that  NlOO  and  N400  have  cortical  sources,  while 
P300  seems  to  have  a  subcortical  source.  Our  data  also  suggest  a 
subcortical  source  for  N200. 

This  totally  unexpected  finding  of  N400  in  an  odd  ball 
experiment  led  us  to  explore  some  of  the  factors  that  might 
affect  this  phenomenon.  After  testing  several  hypotheses,  we  hit 
upon  the  idea  of  varying  stimulus  intensity.  Thus  far  only  two 
levels  of  visual  and  auditory  stimulus  intensity  have  been  tried. 
At  one  level  the  stimuli  are  detectable,  but  are  only  slightly 
above  threshold.  This  leads  to  the  obliteration  of  the  N400  in 
the  electrical  response  (magnetic  data  have  not  yet  been 
evaluated).  At  the  second  level  the  stimuli  are  well  above 
threshold  (at  least  dB) .  This  leads  to  the  presence  of  N400 
in  the' odd  balls,  whether  they  are  visual  or  auditory.  Also, 
these  same  odd  balls  produce  magnetic  N400s  having  a  cortical 
source. 

It  is  now  possible  to  speculate  on  the  temporal  sequence  of 
these  components  of  the  ERP.  The  very  earliest  components  arise 
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in  the  primary  sensory  areas  and  may  or  may  not  reflect  some 
gating  effects  due  to  attention.  NlOO  definitely  reflects  an 
effect  of  cortical  processing,  and  its  magnitude  depends  upon 


attention  to  the  signal  being  processed.  This  processing  is 
further  reflected  in  the  modulation  of  EEG  activity  in  at  least 
some  portions  of  the  sensory  regions  of  the  brain.  P200 
indicates  the  arrival  of  processed  information  at  subcortical 
nuclei,  probably  the  limbic  system.  P300  may  well  indicate  the 
transfer  of  information  between  long  term  and  short  term  memory. 
The  effect  of  this  is  the  transfer  of  information  back  to  the 
cortex  <N4CfO)  for  additional  action.  This  sketchy  view  of 
sequencing  is  consistent  with  many  ideas  expressed  by  others,  and 
it  provides  no  indication . of  the  actual  nature* of  the  processing 
involved.  By  employing  other  more  comp 1 i cat ed- paradigms, 
together  with  our  multi-sensor  array,  we  may  well  gain  a  much 
deeper  insight  into  the  processes  involved. 


